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Abstract: Education in food safety is a well-recognized health intervention, which allows 
the prevention of a wide range of diseases. Among the strategies of control and prevention 
of foodborne diseases, it is indicated that food safety education has the double advantage of 
having low costs and high potential effectiveness, as long as it is carried out with the active 
participation of food handling workers. In many countries, the Food Code has made 
compulsory the sanitary training of food workers. Flowever, like in many other disciplines, 
food science educators receive minimal training on instructional techniques before 
becoming teachers. One of the important questions of the problem here presented is the 
issue related to the methodologies of pre-evaluation and final evaluation. We describe two 
indices to validate the training in food safety, which could be used for the quantification of 
educational intervention. The results show that a better learning process involves the active 
participation of both the students and the educators. We concluded that the evaluation 
process is more complex than the single instance of accreditation though a final evaluation. 
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1. Introduction 

Worldwide, most of the reported foodborne disease outbreaks start from food prepared outside 
home, in both public and private foodservice establishments. It is widely recognized that inadequate 
food preparation practices and food service of community kitchens and dining rooms are strongly 
associated with poor microbiological quality. Education in food safety is a well-recognized (although still 
underutilized) health intervention which allows preventing a wide range of diseases with different 
etiologies [1]. For example, differences between restaurants in which outbreaks had occurred and those 
in which no outbreaks had occurred showed that a qualification in the kitchen management, including 
training in food hygiene, was the only factor that influenced the results [2]. 

Therefore, among the strategies of control and prevention of foodborne diseases, it is indicated that 
food safety education has the double advantage of having low costs and high potential effectiveness, 
as long as it is carried out with the active participation of food handling workers [3,4]. However, like 
educators in many other disciplines, food science educators receive minimal training on instructional 
techniques before becoming teachers [5]. 

Among pedagogic methodologies, test analysis techniques examine how the test items are 
performed as a set, whereas item analysis techniques investigate the performance of items considered 
individually, either in relation to some external criterion, or in relation to the remaining items on the 
test [6]. However, our experience indicates to us that some of the so called best practices in item and 
test analysis are not frequently used in food worker training design. Classically, test analysis refers to 
the application of statistical methods for the characterization of each of the items. 

The aim of the present work was to analyze alternatives for the evaluation of the training process of 
food workers. We also aimed to find out whether only one exam is enough as a tool for the 
accreditation of this evaluation. To this end, two instances of evaluation were analyzed and some of 
their properties were quantified to find a criterion to improve them as training tools. In addition, 
we compared these two instances of evaluation in order to analyze the results obtained through the 
training process by the item analysis use. We found tools to provide food safety educators with good 
criteria to evaluate the performance of training activities. 

2. Methods 

2.1. Train ing A ctivity 

The training activity is a full course on food handling topics. The methodology used for the 
development of the course was a series of educator-trainee sessions with workshop modality. 
The complete activity consisted of eight sessions. The subjects approached considered different aspects 
ranging from the particular rules of food hygiene to general topics in food security. Following previous 
experiences, different practical activities and the work with a reading guide were part of the course. 
Interactive dissertations consist of the topics’ exhibition for the teacher and the active incorporation of 
personal experiences, additional concepts, questions and doubts of the workers. 

The sample consisted of 101 workers who had participated in interactive dissertations on the training 
for food handlers between 2005 and 2007 (42% men), aged 16-78 years old (average: 23 years old). 
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The participants had varying degrees of literacy, and their education ranged from incomplete primary 
education to complete tertiary education. 

The activities were developed in groups of a maximum of 20 people, with different levels of 
experience in food handling. The literacy level varied from a low level and little knowledge on food 
production or elaboration outside home to groups with professional skills. Only 18% were 
active workers, meaning that a few of them were employees with current and permanent activity in 
establishments of food production. The tested groups were not discriminated, taking the total number 
of students independently of their abilities. 

Audiovisual resources designed to explain the topics were present in all the activities. 
These resources included videos, slides, illustrations and cartoons. Some hands-on activities like the 
use of thermometers, the washing of hands, the detection of microbes and the direct observation of 
parasites were also developed. 

Two evaluations were carried out, one at the beginning and the other at the end of the course. 
They were called pre- and post-evaluation, respectively. The same kind of questions was used in the 
pre-evaluation and the post-evaluation test, being answered through multiple choice options. 
All the questions were developed based on previous own experience in food handling hygiene courses. 

2.2. Exam Structure and Item Analysis 

The set of questions for the pre-evaluation and the post-evaluation tests included the same topics, 
grouped into five areas: (a) general concepts of food hygiene; (b) storage and preservation of food; 
(c) acquisition, preparation and foodservice; (d) worker hygiene and (e) infrastructure hygiene. 

The exam had a total of 51 multiple-choice questions (26 of them in the pre-evaluation test, 
and 25 in the post-evaluation test), including mostly primary concerns in food hygiene. Each one of the 
questions had five options to choose as an answer. In order to test specific aspects, some questions 
were about minor topics which were also discussed during training activities. The designs of questions 
considered the previous experience looking to achieve a good evaluation, including also some test 
distracting. Finally, some questions were repeated in both tests. 

The students were all scored in a percentage scale, with Equation (1), where Gi is the number of 
questions answered correctly by student i, and T is a total number of questions. Then, Qi is the 
percentage of questions answered correctly by each student, which means that the Qi for student i is 
his score over 100 points. 

Qi = y* 100 ( 1 ) 

In the test analysis techniques, two properties are frequently considered for the characterization of 
the questions: the Facility Value (Fv) and the Discrimination Index ( Di ). 

The Fv quantifies the possibility of a question to be correctly answered by students. The item 
difficulty is simply the percentage of students taking the test who have answered it correctly. 
The algorithms to calculate the Fv are direct and intuitive. When all the items of the test are extremely 
difficult, the great majority of the test scores will be very low. When all the items are extremely easy, 
most of the test scores will be extremely high. In either case, test scores will show very little variability 
and will not represent real situations or differential behavior of students. That is why items with the 
highest and lowest EVs are recommended to be removed. 
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( 2 ) 


In our work, the Fv for a question is a difficulty index which varies between 0 and 1, being 1 the 
value assigned to a question with maximum difficulty. The Fv for each question is described according 
to Equation (2), where Sc is the number of students who answered the question correctly, and n is the 
total number of students. 
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( 3 ) 


The Di quantifies the possibility that a question be correctly answered by the student that obtains a 
good score and, simultaneously, wrongly answered by a student that obtains a low score. This index 
shows the selectivity of a particular question. A question with high capacity of discrimination between 
the students was assigned with a high value of Di, whereas a question with low Di was assigned to a 
relatively confusing question, whose result could not be clearly associated with the general result of 
the evaluation. There are some algorithms to calculate discrimination indicators, based on statistical 
criteria and are generally not intuitive. High values of discrimination indices are recommended to be 
used in all the items. 


Di = 


i a 


n 


( 4 ) 


In our work, the Di for a given set of exams was determined by the range between the highest and 
lowest score obtained in the group of students evaluated. According to the algorithm developed by 
Corica [7], Di could be described by Equation (3), where Ci is the Corica index for each student and 
each question, and n is the total number of students. 

The final value of Di is described by Equation (4), which means that the value for Di for a single 
question is the Ci mean over all students («). 


3. Results and Discussion 


Adult training implies the use of different tools that contribute to the development of participation 
instances. The tools should not only take into account the previous experience of the students, but also 
govern the activities of the educators. In this sense, the incorporation of the experience of the student 
in food handling is one of the most important tools in the development of training activities [8]. 
Also, many researchers [9] have evaluated how individual and organizational factors influence a range 
of immediate outcomes (/.<?., worker’s knowledge, worker’s self-efficacy, worker’s competencies, 
and commitment to overcome organizational barriers) and ultimate outcomes (i.e., safe food handling 
practices, inspection scores, incidence of foodbome diseases). 

Consequently, for a training program to be effective, it should be based on appropriate adult 
education theory and should incorporate some activities which support the development of relevant 
skills to real life situations, where workers can put the information into practice [10,11]. 

Significant differences have been observed in the understanding and acquisition of concepts, 
when the exams at the beginning and at the end of the courses are compared with pre- and post- 
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evaluation techniques. Literacy is defined not only by someone’s writing and reading abilities, but also 
by the proper understanding and transmission of his own reality. 

After carrying out training activities, such differences have been found to be related to variations in 
both the previous knowledge and literacy of the evaluated students [12]. In many countries, especially in 
those of low and middle income, marginal literacy of workers of food production establishments is a 
common circumstance. This is due to the fact that the education degree is not a condition for their 
employment. Moreover, a great amount of food is produced in non-formal establishments without state 
habilitation. Thus, we postulate that in order to generate effective actions in sanitary policy, 
the initiatives of training must consider these previous circumstances. However, contrary to this point 
of view, other authors suggest the use of homogeneous training codes [13]. 

It has been observed that workers with many years of working experience in the food sector or with 
previous food safety education give correct answers more frequently than those without [14]. In this sense, 
the workshop in which the best interaction between the individuals is verified, the knowledge of the 
group is benefited in two different ways. That means it is benefited by the previous individual 
experience of each student and by the concepts developed by the educator. 

In Argentina, the sanitary training of workers which develop their tasks in direct contact with foods 
has been made compulsory, by the Argentinean Food Code since 2000. However, each province has 
regulated it with different levels of attention. Generally, Latin-American regional laws only emphasize 
the contents of food hygiene and the qualification degree of the educators. They should be specialized in 
food technology and food hygiene without the requirement for training on instructional techniques. 

Nevertheless, as it has been previously discussed, the training of adults requires not only flexible 
contents that consider the methodology with which the specific concepts are developed but also other 
contents that articulate those specific concepts with the daily tasks of food handlers. As a result, 
the task of educators is more than the single presentation of contents. They have to make use of 
suitable pedagogic tools. This implies the handling of techniques of social communication and 
carrying out a previous diagnosis of the group to be trained [15]. 

Unfortunately, neither the aspect of the flexibility of contents nor the aspect of the pertinent 
methodologies is currently taken into account seriously. This is also observed in the regulations applied 
by official agencies that force the training of food handlers. This aspect is clear, for example, in many of 
the training manuals published by the State Health Authority of Buenos Aires, Argentina [16,17]. 
At present, this situation turns the norm useless. 

Most studies assess the acquisition of knowledge of workers by analyzing pre- and post-training 
methods [18]. One of the most important aspects of the problem is the methodologies of diagnosis and 
of final evaluation. Is it possible to evaluate changes in knowledge, behavior, attitude and practices at 
the workplace? The written examination has been profusely used as an accreditation tool of food 
workers [19]. However, this tool mainly evaluates knowledge acquisition. 

Nevertheless, what really allows a better evaluation of the results of the training is the follow-up of 
the changes in behavior in the effective accomplishment of sanitary procedures [3]. For this reason, 
it is necessary to find out the best validation method for an adequate written examination as the only 
accreditation tool [20]. 

In our knowledge, the general idea is to consider that a final written examination should be 
compulsory as a supporting document of the achieved training. Nevertheless, how this exam should be 
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made is not specified far beyond the fact that it is a crucial document. This is true for example in the 
national application of the 21° Article of the Argentinean Food Code. 

The training of food handlers in food safety should not be seen only as a simple training in 
techniques and procedures for the sanitary maintenance of food hygiene. It should also be considered 
under global concepts, including the social context and networks that support the practices, 
the insertion of the workers in the productive system, and the recovery of their role as learning subjects 
in a non-formal system. 

The ability of the current food safety training programs to change behavior has been questioned. 
To increase the effectiveness of these programs, it is important to consider the current behavior of food 
handlers and to understand how this behavior interacts with their beliefs and the level of knowledge [21]. 
From a critical point of view, it is necessary to reflect on how valid and precise a way of transmission is 
to accredit the acquisition of ability and knowledge. Thus, it is important to analyze the relevance of 
the evaluation and accreditation methodology. 

In this work, the scores (Qi) of the post-evaluation test were between 25 and 92, with an average of 
67 and a standard deviation of 14.4, whereas those of the pre-evaluation test ranged between 19 and 73, 
with an average of 46 and a standard deviation of 13.9. The differences between the results of 
the post-evaluation and those of the pre-evaluation (A qi) showed a value of 28 with a standard 
deviation of 16.4. 

The item analysis of the different questions was carried out through Fv and Di values. Table 1 
shows the questions with extreme Fv and Di values for the pre- and post-evaluation tests. 

In regards to the performance of the set of questions, there was a linear and positive correlation 
between both indices in the pre-evaluation test (Figure 1). The Fv index ranged from 0.13-0.86, 
with an average of 0.54 and a standard deviation of 0.18, whereas the Di index ranged from 0.48-0.76, 
with an average of 0.61 and a standard deviation of 0.08. 

In contrast, in the post-evaluation test, there was a clear linear and negative correlation between 
both indices for the set of questions considered (Figure 2). The values of Fv ranged from 0.06-0.75, 
with an average of 0.30 and a standard deviation of 0.20, whereas the values of Di ranged from 
0.44-0.70, with an average of 0.62 and a standard deviation of 0.07. 

Scores obtained in our post-evaluation data set reflect a strong educational intervention. A difference 
of almost 30 points in average was found in the performance of the students when compared with the 
pre-evaluation scores. However, Fv was notably different between the pre- and post-evaluation, 
whereas Di was not. It is argued that as Fv and Di are closely related, there is a better discrimination 
capacity with medium facility values (0.4-0.6). This relation can offer a lot of information about the 
competence among students [22], 

We also observed an important modification in the linear relationship of Di and Fv (Figures 1 and 2). 
Whereas the representation corresponding to the pre-evaluation shows a positive correlation between 
both indices, with an important dispersion in Fv, the representation of the stage of the post-evaluation 
shows a clear linear and negative correlation. 
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Table 1. Selected questions for the pre- and post-evaluation activities, with a respective 
qualification on Facility Value (Fv) and Discrimination Index (Di). 


Question 

Choices 

Pre- 

Post- 

Fv 

Di 

Fv 

Di 

Which is the most 

(a) Contamination by insects and rodents; 





frequent cause of 
transmission of 

(b) Contamination by poor handling of kitchen utensils; 

(c) Poor hand washing; 

0.86 

0.51 

0.69 

0.49 

foodborne diseases 

(d) Poor general personal hygiene. 





Why is it necessary to 

(a) Because it is easier; 





keep waste recipients 

(b) Because waste is a contamination source; 

0.20 

0.53 

0.06 

0.69 

closed and far from the 

(c) Because waste produce bad smell; 

handling food zone? 

(d) Because waste near the food is not adequate. 





Raw eggs ... 

(a) Should be washed and brushed before storage; 

(b) Should be washed and brushed before use; 

(c) Should be disinfected before use; 

(d) Should be stored and used without treatment. 

0.86 

0.55 

0.75 

0.44 







(a) With hot water and soap; 

(b) With hot water and soap, then drying hands with 





The best way to wash 

a dishtowel; 

0.35 

0.49 



your hands is ... 

(c) With hot water and liquid soap, then drying with 

one-use paper; 

(d) With cold water and disinfectant soap. 






(a) To purchase the larger ones in order to assure 
food quality; 





To purchase tins, 
it is important: 

(b) To verify the sell-by date and that they do not 
have dents; 

(c) To always purchase products from large companies; 

(d) To be sure to keep products refrigerated. 

0.13 

0.51 




(a) Cleanliness is not important, but the refrigerator 





How should foods be 
kept in a refrigerator? 

must be cold; 

(b) Raw meat underneath or far from cooked meat; 

(c) So as to have easy access to dried foods; 

(d) Checking that the temperature is right. 

0.72 

0.58 



Aluminum recipients 
should not be used ... 

(a) To preserve soups, stews or fruit juices which are to 
be used the next day; 

(b) To cook food with a high amount of sugar; 

(c) To prepare dough or other foods like bread; 

(d) To dress salads. 



0.06 

0.68 


Which of the following 
sanitary practices are 
useful in the kitchen? 

(a) Do not clean until all the dirt is dried up; 

(b) Use the equipment as least as possible; 

(c) Cleaning while working; 

(d) To keep waxed floors. 



0.07 

0.69 
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Figure 1. Relationship between Fv and Di in pre-evaluation questions. The linear 
correlation found shows a positive slope of 2.46432 (±0.17179) and a coordinate to the 
origin of-0.95341 (±0.10476). r 2 = 0.895540; SD = 0.06544; p < 0.0001; N = 26. 



Figure 2. Relationship between Fv and Di in final evaluation questions. The linear 
correlation found shows a negative slope of 3.0363 (±0.20208) and a coordinate to the 
origin of 2.11685 (±0.1213). r 2 = 0.92698384; SD = 0.06278 ;p < 0.0001; N = 25. 



The negative slope in the linear correlation obtained in the set of questions of the post-evaluation 
stage indicates that the greater discrimination capacity and therefore the higher evaluation value could 
appear in the easiest questions, i.e., in those with small indices of difficulty. In a direct interpretation, 
we observed that the increase in the individual scores obtained in the final evaluation with respect 
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to those obtained in the pre-evaluation stage are related to an individual assimilation of both the 
theoretical and practical concepts. 

Nevertheless, considering the questions posed in both stages, the correlation founded between both 
indices was improved. This could reflect an integral modification of the ideas incorporated from the 
interactive dissertations along the course. 

It is observed that, before the interactive dissertations, the students approached the subject from 
a multiplicity of directions, derived from previous and personal experiences. In contrast, at the 
evaluation stage, they showed a common directionality to answer the questions, which was reflected in 
the correlation obtained. If compared with previous studies carried out by others, our studied group 
was trained during a longer period [18,23]. This may indicate a harder formative intervention in the 
courses reported in our study influencing the assimilation of the knowledge and this should be taken 
into account to obtain objective and quantitative indices. 

It is interesting to point out that the least difficult questions should be selected to define a set of 
questions with a high level of evaluation capacity during the post-evaluation stage. On the other hand, 
the hardest questions are not ideal to discriminate between better or worse trained students. In other 
words, the evaluation stress, associated with the need to accredit the knowledge and abilities acquired, 
does not need to be enhanced by the difficulty in the questions. This is because this difficulty does not 
contribute to the purpose of the evaluation, which is exactly to accredit this knowledge and abilities. 

These results allowed us to design better pre- and post-evaluation tests, including objective 
parameters. We can describe a good pre-evaluation as that which includes a set of questions with a wide 
range of difficulty. The best value for the Fv index does not exist. Besides, a good post-evaluation test 
includes a set of questions with a high index of discrimination and a low to intermediate difficulty. 
Similarly, although ideal maximum values were preferred for Di , the best value for this index for an 
individual question does not exist either. 

Then, we can use these tools to eliminate questions that do not meet minimum standards for 
difficulty and discrimination in both the pre- and post-evaluative stages. The final objective is to have 
better pre- and post-evaluation tests for each group of workers after an iterative methodology. 
This is finally an alternative procedure for testing and validation of evaluation instances. 

The present study was performed within the framework of an activity defined by the requirements 
of the training of workers in the area of food handling. However, the parameters defined in this work 
can be generalized in other non-formal health training contexts. That is, they can be used to quantify 
the utility of the choose question for the pre- and post-evaluation of courses in which a modification in 
the concepts of the students in relation to a structured base of practical knowledge is desired. 

The difficulty and discrimination indices used in the present work were good tools in this context. 
In this way, the marginal literacy of some participants can call into question the efficacy of the indices 
to establish changes in the participants’ knowledge. Nevertheless, these indexes are in use for 
qualifying the groups as a complex entity, and not only by making an individual evaluation of each 
worker. Therefore, the behavior of the group is first validated and then, if it is adapted, the individual 
behavior. At this point, it is necessary to mention that the workers generally recover their tasks in a 
group. In this way, they also reach or not the desired quality standards for the products that 
they prepare. 
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Without doubt, food safety training increased knowledge and improved attitudes about hand 
hygiene practices. However, very few studies have been conducted to evaluate the impact of food 
safety training on food handlers’ attitudes about good hand hygiene practices [24]. Our indices are not 
directly related to the changes in sanitary behavior of workers; it is necessary to find additional 
information for this essential evaluation. 

The learning process that involves the active participation of both the students and educators in 
workshops allow us to consider the evaluation as a process of greater complexity than the single 
instance of accreditation by a final evaluation. This work is an approach to consider an integral point 
of view on the accreditation aspect, incorporating the use of objective evaluation indices of the training 
activity performance. This fact is not exclusive of other accreditation conditions. 

4. Conclusions 

The quantification of the properties of the questions used in the pre- and post-evaluations stages of 
training courses of food handling workers showed a remarkable and opposite correlation between 
discrimination and difficulty indices. 

This phenomenon allows establishing an objective quantification of the degree of educational 
intervention over the groups of students. On the other hand, it allows validating the use of the proposed 
indices for this purpose. This might be used to qualify the performance of any training intervention in 
similar contexts, independently of the applied methodology. 

We propose a methodology for a gradual selection of better questions for a pre- and post-evaluation 
for each group of food workers in an iterative procedure. That is a way to find a better methodology on 
sanitary food initiatives. 

Finally, the heterogeneity of the groups studied and the forcefulness of the correlations obtained 
allow us to propose this method as a more general tool in the evaluation of the performance of adult 
training activities. Though, the methodology here discussed is an important general issue for any type 
of work training, and it is particularly significant in the application of food workers training because 
there is a particular context. That context includes: heterogeneous literacy conditions of participants, 
presence of workers with food experience on the subject, their role as sanitary agents, and how the 
relevance of training results in public health quality. 
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